Adjust read window based on used memory #1013
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
monthonk
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
monthonk
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
monthonk
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
monthonk
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
monthonk
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
monthonk
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
monthonk
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
monthonk
force-pushed
the
mem_limiter
branch
from
1e69a7f
to
250fbb5
Compare
monthonk
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
monthonk
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
monthonk
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
monthonk
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
monthonk
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
monthonk
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
monthonk
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
monthonk
force-pushed
the
mem_limiter
branch
from
250fbb5
to
173aa5d
Compare
monthonk
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR benchmarks
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR benchmarks
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR benchmarks
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR benchmarks
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR benchmarks
GitHub Actions
Inactive
monthonk
force-pushed
the
mem_limiter
branch
from
173aa5d
to
f191fac
Compare
Signed-off-by: Monthon Klongklaew <[email protected]>
monthonk
force-pushed
the
mem_limiter
branch
from
5db1220
to
8fad2e5
Compare
monthonk
temporarily deployed
to
PR benchmarks
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR benchmarks
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR benchmarks
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR benchmarks
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR benchmarks
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
monthonk
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
monthonk
added a commit
to monthonk/mountpoint-s3
that referenced
this pull request
Oct 2, 2024
We have changed some log level in awslabs#1013. We probably want to push some of them down to trace instead of debug. Signed-off-by: Monthon Klongklaew <[email protected]>
github-merge-queue bot
pushed a commit
that referenced
this pull request
Oct 2, 2024
We have changed some log level in #1013. We probably want to push some of them down to trace instead of debug. Signed-off-by: Monthon Klongklaew <[email protected]>
rajdchak
pushed a commit
to rajdchak/mountpoint-s3-fork
that referenced
this pull request
Oct 4, 2024
* Mem limiter prototype Signed-off-by: Monthon Klongklaew <[email protected]> * Clean up development logging Signed-off-by: Vlad Volodkin <[email protected]> * Scale up atomically, scale down after data was consumed Signed-off-by: Vladislav Volodkin <[email protected]> * Remove Client from MemoryLimiter, document this structure Signed-off-by: Vladislav Volodkin <[email protected]> * Simplify the logic and include client metrics Signed-off-by: Monthon Klongklaew <[email protected]> * Correct client mem usage stats Signed-off-by: Monthon Klongklaew <[email protected]> * PR comments Signed-off-by: Monthon Klongklaew <[email protected]> * Put the cli argument behind a feature flag Signed-off-by: Monthon Klongklaew <[email protected]> * Fix scaling logic and address comments Signed-off-by: Monthon Klongklaew <[email protected]> --------- Signed-off-by: Monthon Klongklaew <[email protected]> Signed-off-by: Vlad Volodkin <[email protected]> Signed-off-by: Vladislav Volodkin <[email protected]> Co-authored-by: Vlad Volodkin <[email protected]> Co-authored-by: Vladislav Volodkin <[email protected]>
rajdchak
pushed a commit
to rajdchak/mountpoint-s3-fork
that referenced
this pull request
Oct 4, 2024
We have changed some log level in awslabs#1013. We probably want to push some of them down to trace instead of debug. Signed-off-by: Monthon Klongklaew <[email protected]>
rajdchak
pushed a commit
to rajdchak/mountpoint-s3-fork
that referenced
this pull request
Oct 8, 2024
* Mem limiter prototype Signed-off-by: Monthon Klongklaew <[email protected]> * Clean up development logging Signed-off-by: Vlad Volodkin <[email protected]> * Scale up atomically, scale down after data was consumed Signed-off-by: Vladislav Volodkin <[email protected]> * Remove Client from MemoryLimiter, document this structure Signed-off-by: Vladislav Volodkin <[email protected]> * Simplify the logic and include client metrics Signed-off-by: Monthon Klongklaew <[email protected]> * Correct client mem usage stats Signed-off-by: Monthon Klongklaew <[email protected]> * PR comments Signed-off-by: Monthon Klongklaew <[email protected]> * Put the cli argument behind a feature flag Signed-off-by: Monthon Klongklaew <[email protected]> * Fix scaling logic and address comments Signed-off-by: Monthon Klongklaew <[email protected]> --------- Signed-off-by: Monthon Klongklaew <[email protected]> Signed-off-by: Vlad Volodkin <[email protected]> Signed-off-by: Vladislav Volodkin <[email protected]> Co-authored-by: Vlad Volodkin <[email protected]> Co-authored-by: Vladislav Volodkin <[email protected]>
rajdchak
pushed a commit
to rajdchak/mountpoint-s3-fork
that referenced
this pull request
Oct 8, 2024
We have changed some log level in awslabs#1013. We probably want to push some of them down to trace instead of debug. Signed-off-by: Monthon Klongklaew <[email protected]>
rajdchak
added a commit
to rajdchak/mountpoint-s3-fork
that referenced
this pull request
Oct 8, 2024
This reverts commit f3bceaa.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of change
Add a mechanism to track memory usage of the prefetcher and adjust its read window size based on available memory.
Relevant issues: #987
Does this change impact existing behavior?
Yes, reads are impacted.
Does this change need a changelog entry in any of the crates?
Yes, mountpoint-s3 changelog will be updated along with documentation in the following PR.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the Developer Certificate of Origin (DCO).